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JITTER REDUCTION IN DIFFERENTIATED 
SERVICES (DIFFSERV) NETWORKS 

5 This claims priority from U. S. Provisional Application No. 60/221,928, titled, Jitter 

reduction in Differentiated Services (DiffServ) networks, and filed9 on July 3 1 , 2000, which 
is incorporated herein by reference. 

Field of the Invention 

1 0 This invention is applicable in the field of real time Internet Protocol (IP) applications 

that use the Differentiated Services (Diffserv) concept [RFC2475] to support various Quality 
of Service (QoS) requirements. 

This application makes reference to published protocol documents and similar 
published references which include but are not limited to the following RFC publications: 
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RFC 791, 792, 1812, 1889, 2386, 2475, 2597, 2598, 2638, and 2702. A complete listing of 
the references used in this text is provided at the end of the text. 

DESCRIPTION OF PRIOR ART 

5 Differentiated Services (Diffserv) architecture [RFC2638], [RFC2475] is a result of 

efforts to introduce a highly scalable scheme to provide QoS over IP networks. In Diffserv, 
Per-flow state is pushed to the edges and the traffic through Diffserv routers is treated on 
aggregate basis. The service differentiation is achieve by means of Differentiated Service 
(DS) field in the IP header and the Per-Hop Behavior (PHB) as main building blocks. At 

10 each node packets are handled according to the PHB invoked by the DS byte in the packet 
header. The PHB defines the externally observable behavior at the node. Two PHBs have 
been defined, the assure forwarding (AF-) PHB [RFC2597] and the expedited forwarding 
(EF-) PHB [RFC2598] . The Diffserv domain will provide to its customer, which is a host or 
another domain, the required service by complying fully with the agreed Service Level 

1 5 Agreement (SLA), SLA is a bilateral agreement between the boundary domains negotiated 
either statically or dynamically. The Diffserv domain will provide to its customer (a host or 
another domain) the required service by complying fully with the agreed SLAs. The SLA 
defines the forwarding service that the customer of the Diffserv domain will receive and, the 
Traffic Conditioning Agreement (TC A) which in twin specifies from one point of view, what 

20 the client of the Diffserv domain should do to achieve the desired service and from another 
point of view specifies what the Diffserv domain service provider will have to do to enforce 
the traffic limits, e.g., throughput, specified by the SLA. The service differentiation is 
achieved by means of Differentiated Services (DS) byte and Per-Hop Behavior (PHB) as 
main building blocks. At each router, packets are handled according to the PHB invoked by 

25 the DS byte in the packet header. Up to date there are two PHBs defined: the expedited 
forwarding type (EF-PHB) and the assured forwarding type (AF-PHB). The EF-PHB 
provides tools to build a low loss, low latency, low jitter and an assured bandwidth, end-to- 
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end service through DS domains [RFC2598] . This service is also known as premium service 
[RFC2638]. The AF-PHB group consist of four PHB classes and it can be used to provide 
quality differentiation related to various quality aspects [RFC2597]. It is also known as 
assured service [RFC2638]. 

5 A preferred service for applications with strict QoS requirements is the premium 

service, i.e. Expedited forwarding PHB. 

In Diffserv there are two types of routers defined, the border routers that interconnect 
the Diffserv domain to other Diffserv or non-Diffserv-capable domains, and intermediate 
routers that connect only to other Diffserv intermediate routers or border routers within the 

10 same domain. Both Diffserv border routers and intermediate routers must be able to apply 
the appropriate PHB to packets based on the Diffserv codepoint A Diffserv border router 
can act as a Diffserv Ingress router or as a Diffserv Egress router. 

A Diffserv Ingress border router is the entry router for incoming traffic and a Diffserv 
egress border router is the exit router of the outgoing traffic. The Diffserv Ingress router is 

1 5 responsible for ensuring that the incoming traffic is conforming to the agreements between 
the domain it belongs to and the domain to which it is connected. A Diffserv egress router 
may perform traffic conditioning functions on traffic forwarded to a directly connected 
peering domain, depending on the agreement between the two domains. It is to be noted that 
the terms ingress border router and egress border router used further in describing this 

20 invention are either routers connected to hosts, or to other border routers of other domains, 
respectively. 

Routing in the current IP can be done either statically or dynamically. In static 
routing the routes to destinations are manually configured and network reachability to these 
destinations is independent of the current state in the network itself. It is a stable, but 
25 inflexible routing scheme due to the necessity of manual reconfiguration of the routes any 
time the topology changes. In dynamic routing the routes are configured via routing 
protocols, and network reachability is dependent on the current states of the network. It is 
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a far more flexible scheme than static routing. In current Diffserv implementations static 
routing is used as the main scheme, and dynamic routing is used as an optional scheme where 
the choice of routes depends on the current load on the network. 

Independent of the routing schemes, the IP packets will be routed based on fields in 
5 the IP header. The IP packets can always contain besides the standard fields one of the IP 
options including Security, Strict Source Route, Loose Source Route, Record Route or 
Timestamp. 

The strict source route option gives a complete path that the IP packets should follow. 
Moreover, every router has to support source route options in forwarded packets. In some 
1 0 cases routers may discard the packets with the source route option set. Usually strict source 
route option is used by network administrators for managing the network, as an element of 
various network diagnostic tools. 

The strict source route option in this proposal will be used as a tool in fulfilling the 
low jitter requirements of the applications with strict quality of service requirements. By 
15 means of the strict source route option the network administrator will specify a strict route 
that will be followed by all the subsequent packets generated by the same user application. 

Jitter in Diffserv 

In general, the applications that will have to be supported by the Diffserv domains 
20 will have widely varying traffic characteristics and service demands, on e.g., delay and jitter, 
varying from best effort applications, e.g., e-mail, to real time applications such as audio and 
video. Moreover, in several real time applications the variation in service characteristics can 
be also significant. For example, interactive audio applications have different service 
characteristics from interactive video applications. The main reason for this fact is related 
25 to the human perception for sound and moving images. The human ear is sensitive to sound 
variations that occur within a few milliseconds, whereas, the human eye can only notice light 
variations that occur after 60 milliseconds. 
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These different service characteristics will also impose different QoS requirements 
on the Diffserv network that supports such applications. Referring to the previous example, 
the QoS Requirement on the Diffserv domain will be the variation in the latency or delay to 
transport the data from an ingress border router to an egress border router. Usually this delay 

5 variation is called jitter. Jitter can be comprehended as the absolute value of the difference 
between arrival times of two adjacent packets, i.e., aj and ai, minus their departure times, i.e, 
dj, di, l(aj-dj) - (ai-di)L 

Expedited Forwarding Per Hop Behavior can be implemented in the Diffserv 
architecture, to provide a low loss, low jitter and an assured bandwidth, end to end service 

10 through Differentiated Services domains. In other words, the EF-PHB can assure that at 
every transit router, the aggregate's departure rate must equal or exceed a configurable rate, 
such that the aggregate's maximum arrival rate is less than the aggregate's minimum 
departure rate. However, this can not always guarantee that jitter will be controlled or 
limited. As an example, reference maybe had to the Diffserv experiments accomplished in 

1 5 the TF-TANT project (see [TF-TANT]), where the instantaneous packet delay variation for 
expedited forwarding packets through only two transit routers can get values as high as 10 
milliseconds. It is to be noted that: 

• Jitter is equal to the absolute value of the packet delay variation (see [DPPMWG]). 

• The measured values are obtained when the transit differentiated service router has 
20 to support both expedited and best effort services. In these experiments additional 

assured forwarding (AF) traffic is not included. However, it is expected that in real 
situations the assured forwarding traffic will have a significant contribution. 
Different Diffserv network parameters may have an impact on the end to end jitter 
performance in a Diffserv domain. These include for example: 
25 • The number of routers that a packet has to travel from an ingress border router to an 
egress border router. Note that on average, the absolute value of the jitter increases 
by increasing the number of routers. 
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• The processing time of a packet. Note that by packet processing time is meant the 
sum of the time that a packet has to wait in the queue and the service time of this 
packet. On average, the packet processing time in a router with a fast CPU will be 
smaller than the packet processing time in a router with a slow CPU. 

5 • The jitter on each transit router that is located in the forwarding path of a packet from 
an ingress router to an egress router. Note that on average, the absolute value of the 
jitter in a router with a fast CPU will be smaller than the absolute value of the jitter 
in a router with a slow CPU. 

• The randomness in choosing the forwarding path of the packets that are belonging 
10 to one application session, e.g., one interactive audio application, Voice over IP 

[VoIP] between the calling user and the called user. Usually the randomness, or 
instability in Internet route changes, is caused if: 

► a router or link fails; 

► performance of a link degrades or improves; 

15 ► a router cycles between different routes to the same destination in order to balance 
the load. 



Experiments 

In order to find out what impact the different Diffserv network parameters mentioned 
20 above have on the end to end jitter performance, three sets of simulation experiments were 
conducted. A simulation tool that is known as Queuing Network Analysis Packet (QNAP) 
[VeSe85] was used. To obtain independent replications, the so called method of independent 
replicas was used. Reference may be had in this context to "Art of computer systems 
performance analysis" by Jain R, published by John Wily and Sons, 1991 and to Jain 91. 
25 In all these experiments, one type of traffic was studied, i.e., Expedited Forward. One 

Diffserv domain was used, similar to one depicted in Figure 4, but in place of 7 routers, 30 
routers were used. In the experiment, IP version 4 described in RFC 791 was used, which 
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has a maximum IP field of 9 hops. However, due to the fact that the IP option field is limited 
to only 9 hops, it was specified that the longest path from an ingress border router to an 
egress border router is a total of 9 hops (routers). Note that this Limitation does not apply 
when instead of IP version 4, the IP version 6 (RFC 2460) is used. In order to be able to 

5 control the processing time of a packet and the jitter at each router, each router was modeled 
by an infinite server which has service time distribution a normalized Gaussian (Normal) 
distribution. The normalization is necessary, since the Gaussian distribution can get negative 
values. The mean value of this distribution identifies the mean processing time of a packet 
and its standard deviation identifies the packet jitter introduced by a router. 

10 It is to be noted that the jitter at each router can be caused by different parameters 

such as the best effort and the AF traffic in the Diffserv domain. Furthermore, it is also 
noted that the parameters (mean and standard deviation) of the normalized Gaussian 
distribution used to specify the packet processing time in each router are calculated at the 
beginning of each simulation run. Moreover, regarding the used forwarding path, two 

1 5 scenarios were distinguished. 

In the first scenario (scenario 1) a percentage of the total packets sent from an ingress 
border router to an egress border router and independent of the application session that they 
belong to, can be forwarded hop by hop in a random way that is following a Uniform 
distribution. This percentage of the total packets that follow a random forwarding path 

20 identifies the instability in Internet route changes. It is noted that each router is used only 
once in the forwarding path. 

In the second scenario (scenario 2), all packets belonging to one session will follow 
one predefined forwarding path from an ingress router to an egress router. As in scenario 1 , 
each router is used only once in the forwarding path. 

25 It is to be noted that in each experiment, the results were average over 100 statistical 

independent sessions. Due to the fact that queues with infinite servers were used, the used 
workload will not influence the obtained jitter results. However, it should be noted that the 
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used workload sent through one ingress border router, is 10 packets per second and has a 
Poissonian distribution. 

The obtained 95% confidence intervals are smaller than ±1 % of the calculated mean 

results. 

5 First experiment set 

In this set the impact of the instability in Internet route changes on the end to end 
jitter performance in a Diffserv network was investigated. In this set of experiments both 
experimental scenarios explained in the subheading "Experiments" above were used. In 
particular, for a percentage of instability routes of 0%, scenario 2 was used. In all other 

10 situations scenario 1 was used. The length of the forwarding path is nine routers. The 
parameters (mean and standard deviation) of the normalized Gaussian distribution used to 
specify the packet processing times in each router are calculated as follows. For each router 
the mean value can be in the range from 5 to 20 milliseconds and the standard deviation can 
have a value in the range of 5% of the mean to 20% of the mean. The probability used to 

1 5 obtain a value from the given ranges has a uniform distribution. Figure 1 depicts the results 
obtained in this experimental set. As seen from Figure 1 , Jitter seems to increase somewhat 
linearly with an increasing percentage of unstable routes used from the total routes available. 
From this figure it is seen that by increasing the percentage of instability in the Internet routes 
from 0% to 100%, the mean jitter increases by approximately 100%. 

20 From this experiment set it can be concluded that the percentage of instability in 

Internet route changes should be kept as small as possible. Static routing can always fulfil 
this requirement. Regarding dynamic routing, it has been shown in various papers, [e.g., 
LaAh99], that the instability in Internet routes within the region of one Internet Service 
Provider (ISP) is very small, since no daily or weekly instability frequency components could 

25 be observed. 

Second experiment set 
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In this set of experiments the impact of the router CPU processing speed on the end 
to end jitter performance was studied. In this set of experiments scenario 2 was used. 
However, depending on the packet processing speed on each router, two types of experiments 
were performed. The CPU processing times of the routers used in the first type of 
experiments, specified as SLOW, are in average 4 times slower than the CPU processing 
times of the routers used in the second type of experiments, specified as FAST. In particular, 
in the " SLOW" experiments, for each router the mean value can be in the range from 20 to 
80 milliseconds and the standard deviation can have a value in the range from 5% of the 
mean to 20% of the mean. In the "FAST" experiments, for each router the mean can have 
a value in the range from 5 to 20 milliseconds and the standard deviation can get a value in 
the range from 5% of the mean to 20% of the mean. The length of the forwarding path is 
varied from 1 to 9 routers. 

From the results depicted in Figure 2 it can be deduced that by increasing the CPU 
processing speed from 1 to 4 times on average, the mean jitter increases approximately 4 
times. It is seen from Figure 2 that Jitter increases generally with an increasing number of 
slow routes are compared with fast routes out of the total used routes. By increasing the 
length of the forwarding path, the end to end mean jitter increases almost linearly. 

From these experiments it can be concluded that in order to reduce the end to end 
mean jitter significantly, it is desirable to choose the fastest and the shortest forwarding path. 

Third experiment set 

In this set of experiments the impact of the jitter generated by one router on the end 
to end jitter performance was studied, and scenario 2 was used. Depending on the produced 
jitter on each router, two types of experiments were performed. The jitter on each router was 
generated and varied by changing the standard deviation of the normalized Gaussian 
distribution that is applied to model the router service time distribution. Two types of 
experiments were performed. In both types of experiments, and for each router the mean 
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value can get in the range from 5 to 20 milliseconds. In the first type of experiments, 
specified as "LOW JITTER", the jitter generated by each router is on an average 2 times 
lower than the jitter generated by each router in the second type of experiments, specified as 
"HIGH JITTER", In particular, in the "LOW JITTER" experiments, for each router the 
standard deviation can have a value in the range from 5% of the mean to 12.5% of the mean. 
In the "HIGH JITTER" experiments, for each router the standard deviation can have a value 
in the range from 5% of the mean to 20% of the mean. The length of the forwarding path is 
varied from 1 to 9 routers. 

Figure 3 generally shows the correlation between end to end mean jitter and the 
number of routers having different degrees of jitter. From the results depicted in Figure 3, 
it can be deduced that by increasing the jitter on the routers 100% on average, the end to end 
mean jitter increases approximately 30%. By increasing the length of the forwarding path, 
the end to end mean jitter is increasing almost linearly. 

From these experiments it can be concluded that in order to reduce the end to end 
mean jitter significantly, the shortest forwarding path that includes routers that generate very 
small jitter have to be chosen. In general, the routers that create the smallest jitter are also 
the fastest routers. 

PRIOR ART SOLUTIONS: 

In general, several solutions have been presented to solve the jitter problem at the application 
or the network level. 

• Solution_l attempts to solve the jitter problem at the application level, i.e., only at 
the communication end points, by using a synchronization buffer. Jitter is removed 
by correlating actual arrival time differences with the corresponding sender time 
differences. This solution can be achieved through the time stamps used in the Real 
time Transport Protocol (RTP). See RFC (1889). 
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The following additional solutions are known to be used to solve jitter at only the 
network level. 

• Solution_2: this mechanism is widely used to control jitter in a network (see e.g., 
[Tan96]). Before sending the packets through the network, each router will have to 
compute the expected transit time. At the time that a packet arrives at a router, the 
router will check if the packet is behind or ahead the scheduled expected transit time. 
This information is stored in the packet and is updated at each hop. The next hop 
that receives the packet will check this stored information and if the packet is ahead 
of the predefined schedule, it will hold the packet long enough to get it back on 
schedule. In a situation where the packet is behind its schedule, the router will give 
it a high priority such that it will be sent as fast as possible to the output. 

• Solution_3: this solution is used in a Diffserv 

domain, where the incoming traffic through one ingress border router is shaped 
according to a predefined Traffic Conditioning Agreement (TCA) (see [RFC2475]). 
By providing suitable traffic shaping, the amount, volume and rate of the traffic, and 
thus also the jitter, sent into the Diffserv domain can be controlled. 

• Solution^: this solution is described in [RFC2386], where an architecture, called 
QoS - routing, has been specified to provide dynamic routing by using QoS 
requirements. The QoS requirements, e.g., jitter are used during the routing process. 
In this way an optimal route (or path) with a minimum end to end jitter can be found. 

• Solution_5 : this solution called Multiprotocol Label Switching (MPLS), and similar 
to Solution_4 is able to use QoS requirements during the dynamic routing process. 

PROBLEMS WITH KNOWN SOLUTIONS 

Solution^! : this solution can only be used at the application level. Therefore, if the 
jitter in a network becomes large, i.e., in a large network such as one that supports a Diffserv 
architecture, this solution will not be able to reduce the end to end jitter satisfactorily. 
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Solution^: Due to its complexity, if the router should implement a complex packet 
scheduling and buffering mechanism, it will not be able to satisfy the Diffserv scalability 
requirements (see [RFC2475]); 

Solution_3 : The disadvantage of this mechanism regarding jitter control, is that it is 
5 only solving the jitter problem at the Diffserv domain border routers, and not into the 
Diffserv domain intermediate routers. The measurements accomplished in the TF-TANT 
project (see Section 3.2) show that this mechanism does not completely satisfy the low EF 
low jitter requirements. 

Solution_4 : This architecture has been specific by an Internet Engineering Task Force 
10 (IETF) Working Group (WG) called QoS routing WG. This WG has studied this 
architecture and it has found that is quite complex to be implemented and provided as a 
general QoS routing solution. 

Solution_5: This solution is specified in the MPLS IETF WG and is the most 
promising. However, this architecture remains complex and the IETF will have to address 
15 several issues (see e.g., [RFC2702]), before its future can be decided. 

A need still exists for a method of efficiently reducing jitter in Diffserv networks in 
the context of an access or core network, regardless of whether static or dynamic routing is 
used. 



20 SUMMARY OF THE INVENTION 

The present invention offers a method/ algorithm for reducing jitter and can be 
viewed as an entirely new solution in the area of Diffserv networks. 

This method/algorithm can be applied in any Diffserv domain, in an access or core 
network, regardless of which routing mechanism is used, i.e., static or dynamic routing. 
25 The invention in its broad form resides in a method of reducing jitter in an Internet 

Protocol (IP) packet transmission in a network which uses a strict source route option, a 
forwarding cache and IP destination address for user data packets, the network having at least 
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one ingress border router and an egress border router and an intermediate router said method 
comprising the operational steps in the ingress border router, intermediate routers and 
egress border router generally as in figure 5. 

In a preferred embodiment, the following operational steps are included: 

5 Operation steps in the ingress border router: 

Step_Al: the ingress border router, based on the Traffic Conditioning Agreement 
(TCA) recognizes the EF packets with firm jitter requirements. This operation is similar to 
what is specified in [RFC2475]. Additionally, this ingress border router should contain and 
maintain a list with the shortest and the fastest forwarding paths to each egress border router. 
1 0 This information has to be stored and managed by the Diffserv domain provider. Each time 
that the ingress border router classifies an EF packet and it has no forwarding cache entry for 
its IP destination address,( i.e., the first sent packet of a certain application running on a 
host,) it then extracts by using special filters configured in the ingress border router, the 
shortest and fastest path to an egress border router that is best matching the IP packet 
1 5 destination address. If the ingress border router has an entry for its IP destination address 
then Step_A4 should be followed. 

Step_A2) in the strict source route option it inserts a list of router addresses that 
identify the selected path found in Step__Al . 

Step_A3) the next hop IP address for this particular IP destination address will be 
20 stored in its forwarding cache. 

Step_A4) all subsequent packets to the same IP destination address will be forwarded 
to the next hop IP address stored in its forwarding cache. Step Al , Step_A2 and Sstep_A3 
do not apply for these packets. 

Step_A5) Depending on the size of the forwarding caches and on the types of 
25 applications that the Diffserv domain will support, e.g., VoIP, the Diffserv Domain operator 
will have to define a time period, say T_update, that will be used for updating the selected 
path. In other words, the router will repeat the steps Step_A2, Step_A3 and Step_A4 each 
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T_update may be set to 60 seconds. Note that the mean duration time of a Public Switched 
Telephony Network (PSTN) varies from 120 to 1 80 seconds. Moreover, the activation of 
the updated time period, T_update ? is set by the ingress border router at its start up. 
Therefore, each T_update period all EF packets with firm jitter requirements, that can be 
5 characterized as being of the same application type, will be sent by the ingress border router 
using the strict source route option and inserting a list of router addresses that identify the 
selected paths found. 

The operational steps explained above are illustrated in Figure 5. 

10 The optional operation steps in the interior routers in a preferred embodiment include: 

Step_B 1 : All intermediate routers that receive a packet with the strict source route 
option set will forward this packet to the first router address in the strict source route option 
list, which is the net hop for this packet. The next hop address will remain in the routers 
forwarding cache table. 

1 5 Step_B2: All the subsequent packets that have the same IP destination address will 

also follow the same path that the first packet has followed. Note that all these subsequent 
packets will have the strict source route option turned OFF. 

Operation step in the egress border router: 

20 Step__Cl : When the router receives a packet in which this list is completed, i.e., the 

egress border router, than the packet has reached its destination. 

A modification of the inventive method includes the step of maintaining a list of the 
shortest and fastest forwarding paths which are configured by each border router. 

A variation of the invention includes the step of selectively updating the selected 
25 shortest and fastest forwarding paths within a predetermined time period. 
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A further variation of the invention includes the step of choosing the predetermined 
time period based on the forwarding cache size and types of applications which the network 
will support. 

The invention also includes a software program and a programmed memory to 
execute the above method. 

The invention in another form resides in a method of reducing jitter in Internet 
Protocol (IP) packet transmission in a Differentiated Service Architecture (Diffserv) network 
with assured packet forwarding, said Diffserv using a forwarding cache and IP destination 
addresses for packets, and an on demand or traffic driven route cache population, the network 
having at least an ingress border router and an egress border router, comprising the steps of: 

actuating an IP forwarding optional parameter using a source route option; 

receiving a packet which has a specified IP designation address; 

checking the forwarding cache if an entry is available for the IP destination address; 

if affirmative, sending a received packet to next hop specified by the forwarding 
cache entry; 

if there is no entry in the forwarding cache, then, checking if a route table entry exists 
for the specified destination address; 

if there is a route table entry, then, storing the route table entry into the forwarding 
cache, and sending the packet on its way; 

if there is no forwarding cache entry for the IP destination address, then selecting and 
extracting using filters the shortest and fastest path to an egress border router; 

inserting in the source route option a list of the router addresses that identify the 
selected path; 

storing in the forwarding cache a next hop IP address for the IP destination address; 
sending the received packet via the selected path; and 

handling following packets which follow the received packet and are bound to the 
same destination address by routing them to follow the received packet. 
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In a modification, the Diffserv network includes a strict source route option and 
intermediate routers which receive a further packet with the strict route option set, the 
method including the step of forwarding further packet to a first router address in the strict 
source route option list which is the next hop for said further packet. 

5 The invention also consists in a method of reducing jitter in an Internet Protocol (IP) 

transmission in a Differentiated Service Architecture (Diffserv) network which handles 
information packets with expedited forwarding (EF), said network using a forwarding cache, 
a strict source route option, at least one ingress border router and one egress border router, 
and a Diffserv domain provider, said method comprising the steps of: 

10 a. recognizing EF packets which have firm jitter requirements; 

b. storing and maintaining in the ingress border router using the Diffserv domain 
provider a list of the the shortest and fastest forwarding paths from the ingress border router 
to each egress border router; 

c. ascertaining if a received EF packet has no entry for its IP destination address, and 
15 if affirmative; 

d. extracting using special filters configured in the ingress border router, path- 
information regarding a selected path representing the shortest and fastest path to an egress 
border router that best matches the IP destination address of the EF packet; 

e« inserting in the strict source option, a list of router addresses which identify the 
20 selected path in step (d); 

£ storing in the forwarding cache a next hop IP address for the particular IP destination 
address; and 

g. forwarding subsequent IP packets subsequent to said EF packets to said next hop IP 
address. 

25 A modification includes the step of defining a time period T_update which is used 

for updating the selected path of step d as above. 
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Preferably, the time period T_update is approximately 60 seconds, depending on the 
application. In an embodiment, the time period T_update is used in a Voice Over IP (VoIP) 
application. 

The time period T_update could be used in a Public Switched Telephony network 
(PSTN), where preferably, T_update is in the range of 120 to 180 seconds. 

Advantageously, the method might include the step of setting up activation of the 
time period Tupdate at start up time of an ingress border router. 

The invention also resides in a computer program/memory for reducing jitter in an 
Internet Protocol (IP) packet transmission in a Differentiated Service Architecture (Diffserv) 
network with assured packet forwarding, said Diffserv using a forwarding cache and TP 
destination addresses for packets, and an on demand or traffic driven route cache population, 
the network having at least an ingress border router and an egress border router, the 
program/memory using a method comprising the steps of: 

actuating an IP forwarding optional parameter 

receiving a packet which has a specified IP destination address; 

checking the forwarding cache if an entry is available for the IP destination address; 

if affirmative, sending a received packet to a next hop specified by the forwarding 
cache entry; 

if there is no entry in the forwarding cache, then, checking if a route table entry exists 
for the specified destination address; 

if there is a route table entry, then, storing the route table entry into the forwarding 
cache, and sending the packet on its way. 

The invention also resides in a computer program and a memory for reducing jitter 
in an Internet Protocol (IP) transmission in a Differentiated Service Architecture (Diffserv) 
network which handles information packets with expedited forwarding (EF), said network 
using a forwarding cache, a strict source route option, at least one ingress border router and 



795013 v 1, 34648.00452USPT 



-17- 



Patent Application 
Docket #34648-00452USPT 
EMN2000067 

one egress border router, and a Diffserv domain provider, said program/memory using a 
method comprising the steps of: 

a. recognizing EF packets which have firm jitter requirements; 

b. storing and maintaining in the ingress border router using the Diffserv domain 
provider a list of the shortest and fastest forwarding paths from the ingress border router to 
each egress border router; 

c. ascertaining if a received EF packet has no entry for its IP destination address, and 
if affirmative; 

d. extracting using special filters configured in the ingress border router path- 
information regarding a selected path representing the shortest and fastest path to an egress 
border router that best matches the IP destination address of the EF packet; 

e. inserting in the strict source option, a list of router addresses which identify the 
selected path in step (d); 

BRIEF DESCRIPTION OF THE DRAWING 

A more detailed understanding of the invention may be had from the following 
description of preferred embodiments, given byway of example only, and to be understood 
in conjunction with the accompanying drawing wherein: 

Figure 1 is an illustration of the variation of jitter in microseconds, with the 
percentage of routes which are unstable; 

Figure 2 is an illustration of how jitter varies with the number of fast and slow 
routers; 

Figure 3 is an illustration of the variation of jitter (in microseconds) with the number 
of known high jitter and low jitter routes; and 

Figure 4 is a Diffserv domain example incorporating the invention. 

Figure 5 is a flow diagram illustrating the steps for handling data packets in the 
ingress border router, using the invention. 
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DETAILED DESCRIPTION 

Described hereinafter is a method/algorithrn/memory for limiting jitter in an IP 
transmission. The following description is intended to be understood in light of jitter 
reduction in IP transmission in a Diffserv network. 

Listed below are preferred requirements to be generally satisfied in order to apply the 
invention: 

• Requirement^: In each router, Diffserv domain border and intermediate, the 
possibility of packet forwarding has to be supported. In most router 
implementations, this can be achieved by using a forwarding cache and by activating 
(setting to ON) the optional parameter "ip forwarding" that is used in the operation 
system kernel of each router (see [Stev97]. The "ip_forwarding" option (see 
[Stev97]) is applied to ensure that the IP packets passing a router will be forwarded. 
It is to be noted that the packet forwarding process is usually referred to as on- 
demand or traffic-driven route cache population and it works in the following way 
(see [FeHu98]). When a router receives a packet, it checks the forwarding cache to 
find out if an entry for the IP destination address specified in the packet is available. 
If that is the case then the packet is sent to the next hop specified by the forwarding 
cache entry. When no entry exists in the forwarding cache then the router will check 
if a route table entry exists for the specified IP destination address. If the route table 
has an entry then this entry is stored into the forwarding cache and the packet is sent 
on its way. Otherwise the router sends an Internet Control Message Protocol (ICMP) 
[RFC792] unreachable message to the sender and the packet is discarded. 
• Requirement_2: the ingress border router, based on the Traffic Conditioning 
Agreement (TCA) recognizes the EF packets with firm jitter requirements. 
Additionally, this router should contain and maintain a list with the shortest and the 
fastest forwarding paths to each egress border router. This information has to be 
stored and managed by the Diffserv domain provider. Each time that the ingress 
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router classifies an EF packet (i.e., the first sent packet of a certain application 
running on a host) and it has no entry for its IP destination address, then it will have 
to follow the steps listed below: 

> Step_l):by means of special filters configured in the router it extracts the 
5 shortest and fastest path to an egress border router that is best matching the IP packet 
destination address. 

>■ Step_2) inserts a list of router addresses that identify the selected path found 
in Step_l in the strict source route option. 

5> Step_3) the next hop IP address for this particular IP destination address will 
1 0 be stored in its forwarding cache. 

Step_4) all subsequent packets to the same IP destination address will be 
forwarded to the next hop IP address stored in its forwarding cache. Step_l, Step_2 and 
Step_3 do not apply for these subsequent packets. 

• Requirement _3 : All intermediate routers that receive a packet with the strict source 
1 5 route option set will forward this packet to the first router address in the strict source 

route option list, which is the next hop for this packet. The next hop address will 
remain in the routers forwarding cache table. All the subsequent packets that have 
the same IP destination address will also follow the same path that the first packet 
has followed. It is to be noted that all these subsequent packets will have the strict 
20 source route option turned OFF. 

When the egress border router receives a packet in which this list is completed, then 
the packet has reached its destination. 

• Requirement^!: Depending on the size of the forwarding caches and on the types of 
applications that the Diffserv domain will support, e.g., VoIP, the Diffserv Domain 

25 operator will have to define a time period , say T_update, that will be used for 

updating the selected path. In other words, the router will repeat the steps Step_2, 
Step_3, and Step_4 listed in Requirement^ each T_update time. For a VoIP 
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application, T_update may be set to 60 seconds. It is noted that the mean duration 
time of a Public Switched Telephony Network (PSTN) varies from 120 to 180 
seconds. Moreover, the activation of the updated time period, T_update, is set by the 
ingress border router at its start up. Therefore, for each T_update period, all EF 
packets with firm jitter requirements that can be characterized as being of the same 
application type, will be sent by the ingress border router using the strict source route 
option and inserting a list of router addresses that identify the selected paths found. 
All intermediate routers, will operate as described in Requirement_3. 

Operation 

Considering that the requirements listed above are fulfilled, the algorithm of this 
invention will be able to operate in any Diffserv domain. This means that it will be able to 
operate in Diffserv domain used as an access network or in Diffserv domain used as core 
network (see [RFC2475]), independently of which routing mechanism is used, i.e., static or 
dynamic routing. The algorithm operates as follows. 

Suppose that mixed traffic, e.g., EF, AF and best effort package, is sent to the 
Diffserv domain under study. The ingress border router of this domain will recognize the 
EF packets with specific jitter requirements, classify and process them as specified in 
[RFC2475] and in the Requirement_2 described earlier. 

In other words, using the Traffic Conditioning Agreement (TC A) to recognize the EF 
packets with firm jitter requirements (see [RFC2475]), if the ingress router after classifying 
the EF packet (which may be a first sent packet of a certain application running on a host) 
could not find an entry for its IP destination address, then the router will have to follow the 
steps listed below: 

• Step _1): by means of special filters configured in the router it extracts the shortest 
and fastest path to an egress border router that is best matching the IP packet 
destination address. 
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• Step_2) in the strict source route option it inserts a list of router addresses that 
identify the selected path found in Step_l . 

• Step_3) the next hop IP address for this particular IP destination address will be 
stored in its forwarding cache. 

5 All subsequent packets that have to be sent to the same IP destination address will 

be forwarded to the next hop IP address stored in its forwarding cache. Step_l , Step_2 and 
Step_3 do not apply for these subsequent packets. 

All the intermediate routers that receive a packet with the strict source route option 
set will forward this packet to the first router address in the strict source route option list, 

10 which is the next hop for this packet. The next hop address will remain in the routers 
forwarding cache table. Thus the following packets to the same destination will also follow 
the same path that the first packet has followed. When the router receives a packet in which 
this list is completed then the packet has reached its destination. The routers used for this 
purpose in the Diffserv domain are enabled to specify the source route by means of special 

15 filters implemented in the routers [RFC1812]. EachT_update seconds, e.g., 60 seconds, the 
operation steps described above will be repeated. 
First example 

Figure 4 generally shows a Diffserv domain example where the present invention can 
be applied in this example (see Figure 4) the operation of the algorithm in the Diffserv 
20 Domain 1 is illustrated. This domain is used as an access network. Host X 401 , Host Y 402, 
and Host Z 403 are hosts which among other traffic, can support EF QoS aware applications 
with firm jitter requirements. Router {Rl, R2,...R6} are Diffserv routers. 

Host X sends EF data traffic with firm jitter requirements to Host Y via router Rl . 
In this example Rl is an ingress border router and the router R7 405, is an egress border 
25 router. The router Rl will apply the Requirement_2 explained earlier. In other words, it 
recognizes the first sent packet as an EF packet with firm jitter requirements, and by means 
of special filters and in the strict source route option, it inserts a list of router addresses that 
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the packet sent to Host Y should follow. Note that the Diffserv domain provider has 
predefined this list. For example this list could be routers R3, R4 and R7. In this case Strict 
Source Route option will look like this: 

Rl Strict Source Route option: IPaddrR3, IPaddrR4, IPaddrR7 
Furthermore Rl will create an entry for the IP destination address in its forwarding 

cache. 

Routers R3 and R4 will store in their forwarding caches the next hop address from 

the strict source option list when this packet traverses them. 

When thepacket arrives at router R7,thepointer will point to the destination address, 

i.e. the address of Host Y, which means that it has reached it final destination. This entry 
will be stored by R7 in its forwarding cache. 

All subsequent packets that the host X sends to host Y will be sent by the router Rl 
as normal Diffserv packets, i.e., without activating the strict source route option. Due to the 
fact that the routers Rl, R3, R4 and R7 will have an entry in their forwarding caches for the 
IP destination address of host Y, all subsequent packets that are sent to host Y will follow 
the same path as the path followed by the first EF packet that was sent by the Host X to host 
Y. Each Tjipdate seconds, e.g., 60 seconds, the operation steps described above will be 
repeated. 
Second example 

20 The solution provided for the first example described in the previous Section, is a 

single Diffserv domain solution. 

In the event that hosts are residing in different Diffserv domains (see Figure 4) and 

e.g., Host X sends EF packets with firm jitter requirements to Host Z, then the following 

actions have to be accomplished. 
25 In this example and for the Diffserv domain 1 the router Rl is an ingress border 

router and the router R5 is an egress border router. For the Diffserv domain 2 the router Rl 

is an ingress border router and the router R5 is an egress border router. 
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The operation of the algorithm for the packets that are traversing the Diffserv 
domain 1 will be similar to the operation of the algorithm explained in the first example. 
The only difference is that the egress border router of this domain is router R5 and not R7. 
Furthermore, the predefined list will be R3, R4 and R5. 

5 When the first packet sent from host X to host Z traverses the Diffserv domain 1 and 

it arrives at the ingress router Rl of the Diffserv domain 2, then it will apply the 
Requirement ! explained earlier. In other words, it recognizes the first sent packet as an EF 
packet with firm jitter requirements and by means of special filters and in the strict source 
route option it inserts a list of router addresses that the packet sent to Host Z should follow. 

10 Note that the Diffserv domain provider has also predefined this list. For example this list 
could be: router R4 and R5. hi this case Strict Source Route option will look as follows: 
Rl Strict Source Route option: IpaddrR4, IpaddrR5 

Furthermore Rl will create an entry for the TP destination address in its forwarding 

cache. 

1 5 Router 4 will store in its forwarding cache the next hop address from the strict source 

option list when this packet traverses it. 

When the packet arrives at router R5, the pointer will point to the destination address, 
i.e. the address of Host Z, which means that it has reached its final destination. This entry 
will be stored by R5 in its forwarding cache. 

20 All subsequent packets that the host X sends to host Z will be sent by all router in 

both Diffserv domains as normal Diffserv packets, .i.e., without activating the strict source 
route option. Due to the fact that all the selected routers will have an entry in their 
forwarding caches for the IP destination address of host Z, all subsequent packets that are 
sent to host Z will follow the same path as the path followed by the first EF packet that was 

25 sent by the Host X to host Z. 

Each T_update seconds, e.g., 60 seconds, the operation steps described above will 

be repeated. 
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A significant advantage introduced by this invention is related to the fact that the 
Diffserv domain that implements this idea will be able to reduce the jitter in real time 
Premium Services, e.g., voice, without increasing significantly the complexity in the Diffserv 
domain router. 

5 Figure 5 is a flow diagram for handling Expedited Forwarding (EF) data traffic which 

arrives at the ingress border router. In step Ala, there is recognition of an EF packet with 
firm jitter requirements. In step Alb a determination is made whether the ingress border 
router has a forwarding cache entry for the IP packet destination address. If affirmative, the 
process leads to step A5 which finds out if the T_update period has expired. If A5 is 
10 negative, the process proceeds to step A4. If Alb is negative, the shortest and fastest path 
to an egress border router which is best matching with the IP packet destination address is 
extracted by using special filters in step Ale. Folic wing this, in step A2, a list of router 
addresses is inserted to identify the selected path found in step A7 in the strict source route 
option. In step A3, the next hop IP address for the particular destination address is stored in 
1 5 its forwarding cache. In the final step A4, all subsequent packets to the same IP destination 
address are forwarded to the next hop IP address stored in its forwarding cache. Steps Al, 
A2 and A3 do not apply for these packets. 

The foregoing is a description of the inventive method which offers jitter reduction 
in a data network by recognizing data packets with firm jitter requirements and forwarding 
20 them by the fastest and shortest paths based on the IP destination address of the packet. An 
ingress border router or edge node which recognizes a packet with firm jitter requirements, 
if there is no forwarding cache entry of the destination address, will find and insert a list of 
router addresses satisfying the shortest and fastest path in the strict source IP routing option. 
All intermediate routers receiving a packet with the strict source routing option will forward 
25 the packets tot he next hop specified in the strict source route IP option. The next hop 
address will remain in the routers forwarding cache table. Subsequent packets which have 
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the same IP destination address will follow the same path which the first packet followed; 
the subsequent packets have the strict source option turned off. 



EQUIVALENTS 

5 The invention can be applied in all types of IP Autonomous Systems giving the 

possibility to a router to fix a specific predefined route and use it to send subsequent packets, 
without activating the strict source routing IP option, to the same destination. Note that an 
Autonomous System can be defined as a network wherein the used routing algorithms can 
operate independently of the routing algorithms applied in different networks. 

10 Although preferred embodiments of the method and apparatus of the present 

invention have been illustrated in the accompanying drawings and described in the foregoing 
detailed description, it will be understood that the invention is not limited to the 
embodiments disclosed, but is capable of numerous rearrangements, modifications, 
equivalents and substitutions without departing from the scope of the invention as set forth 

15 in the appended claims. 
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